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(57) ABSTRACT 

A communication device comprises first and second circuits 
to implement a plurality of ports via which the communica- 
tive device is operable to communicate over a plurality of 
communication channels. For each of the plurality of ports, 
the communication device comprises: command hardware 
that includes a first transmitter to transmit data over a respec- 
tive one of the plurality of channels and a first receiver to 
receive data from the respective one of the plurality of chan- 
nels; and monitor hardware that includes a second receiver 
coupled to the first transmitter and a third receiver coupled to 
the respective one of the plurality of channels. The first circuit 
comprises the command hardware for a first subset of the 
plurality of ports. The second circuit comprises the monitor 
hardware for the first subset of the plurality of ports and the 
command hardware for a second subset of the plurality of 
ports. 


17 Claims, 4 Drawing Sheets 
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HARDWARE EFFICIENT MONITORING OF 
INPUT/OUTPUT SIGNALS 

GOVERNMENT INTEREST STATEMENT 

5 

The U.S. Government may have certain rights in the 
present invention as provided for by the terms of contracts 
NNJ06TA25C (Prime) and RH6-118204 (sub -contract) 
awarded by NASA/Lockheed Martin Company. 

BACKGROUND 

In one type of fault tolerant system, for each port, one set of 
hardware, called the command hardware or “COM”, trans- 
mits and receives signals and another independent set of 
hardware, called the monitor hardware or “MON”, monitors 
the signals transmitted and received by the command hard- 
ware. For example, in one such system, a wrap-back mecha- 
nism couples the transmitter of the COM hardware to a 
receiver in the MON hardware. Also, in such a system, for 
each port, each received signal is delivered to two indepen- 20 
dent receivers: one in the monitor hardware and one in the 
command hardware. Thus, for each port, such systems 
require three receivers and one transmitter: one transmitter 
and one receiver in the command hardware and two receivers 
in the monitor hardware. 25 

In some applications, it is especially desirable to imple- 
ment the physical layer of such a COM/MON system using 
similar hardware for both the command hardware and the 
monitor hardware such that they use the same number of 
transmitters and receivers. However, doing so typically leads 
to waste. FIG. 1 illustrates the physical layer portion of a 
COM/MON system that is implemented using the similar 
hardware for both the command and monitor hardware. In the 
example shown in FIG. 1, a device 100 comprises two ports 
105. The device 100 is implemented using two transceiver 
circuits, one of which (also referred to here as the “COM IC” 35 
104) is used to implement the command hardware for each 
port 105 and the other of which (also referred to here as the 
“MON IC” 106) is used to implement the monitor hardware 
for each port 105. Each of the circuits 104 and 106 includes 
four transceivers 108, each of which includes a respective 40 
transmitter 110 and a respective receiver 112. 

The one transmitter and one receiver needed to implement 
the COM portion for each port 105 are implemented using a 
single one of the transceivers 108 in the COM IC 104. The 
transmitter 110 and receiver 1 12 used to implement the COM 
portion for each port 105 are coupled to the transmit portion 
114 and receive portion 116 portion, respectively, of the com- 
munication channel 118 to which that port 105 is coupled. 

The two receivers needed to implement the MON portion 
for each port 105 are implemented using the receivers 112 of 
two of the transceivers 108 included in the MON IC 1 0 6 . One ^ ^ 
of the receivers 112 is coupled to the transmitter 110 for the 
respective COM portion for that port 105 using a wrap-back 
link 120, and the other receiver 112 is coupled to the receive 
portion 116 of the communication channel 118 in order to 
receive data from the communication channel 118. 55 

The design shown in FIG. 1 uses similar hardware to imple- 
ment the physical layer for each of the COM and MON 
portions for each port 105. However, as shown in FIG. 1, for 
each port 105 of the device 100, one entire transceiver 108 in 
the COM IC 104 and two transmitters 110 in the MON IC 106 60 
are wasted (where the wasted elements are marked with an 
“X”). 

SUMMARY 

65 

In one embodiment, a communication device is provided. 
The communication device comprises first and second cir- 


2 

cuits to implement a plurality of ports via which the commu- 
nicative device is operable to communicate over a plurality of 
communication channels. For each of the plurality of ports, 
the communication device comprises: command hardware 
that includes a first transmitter to transmit data over a respec- 
tive one of the plurality of channels and a first receiver to 
receive data from the respective one of the plurality of chan- 
nels; and monitor hardware that includes a second receiver 
coupled to the first transmitter and a third receiver coupled to 
the respective one of the plurality of channels. The first circuit 
comprises the command hardware for a first subset of the 
plurality of ports. The second circuit comprises the monitor 
hardware for the first subset of the plurality of ports and the 
command hardware for a second subset of the plurality of 
ports. 

DRAWINGS 

Features of the present invention will become apparent to 
those skilled in the art from the following description with 
reference to the drawings. Understanding that the drawings 
depict only typical embodiments of the invention and are not 
therefore to be considered limiting in scope, the invention will 
be described with additional specificity and detail through the 
use of the accompanying drawings, in which: 

FIG. 1 is a block diagram of a conventional fault tolerant 
device. 

FIG. 2 is a block diagram of a network comprising a fault- 
tolerant device according to embodiments of the present 
invention. 

FIG. 3 is a block diagram of another network comprising a 
fault tolerant device according to embodiments of the present 
invention. 

FIG. 4 is a block diagram of another network comprising a 
fault tolerant device according to embodiments of the present 
invention. 

In accordance with common practice, the various 
described features are not drawn to scale but are drawn to 
emphasize specific features relevant to the present invention. 
Like reference numbers and designations in the various draw- 
ings indicate like elements. 

DETAILED DESCRIPTION 

FIG. 2 is a block diagram of one embodiment of a network 
201 that comprises a fault-tolerant networking communica- 
tion device 200. In the particular embodiment shown in FIG. 
2, the device 200 is implemented as a networking device 200 
that interconnects a plurality of nodes 203 (individually 
labeled as 203-1, . . . , 203-N in FIG. 2). In the embodiment 
shown in FIG. 2, the networking device 200 comprises at least 
one port 205 for each node 203 that couples the node 203 to 
the networking device 200 via a respective communication 
channel 218. The ports 205 are individually labeled as 
205-1, . . . , 205-N in FIG. 2. In some implementations of such 
an embodiment, the networking device 200 is implemented as 
an Open Systems Interconnection (OSI) International Stan- 
dards Organization (ISO) layer-l/layer-2 networking device 
(such as a hub or repeater) in which any data validly received 
on one port 205 of the networking device 200 is broadcast on 
the other ports of the networking device 200. In other imple- 
mentations of such an embodiment, the networking device is 
implemented as an OSI ISO layer 3 networking device such 
as a bridge, switch, or router, in which any data validly 
received on one port of the networking device 200 is re- 
transmitted on one or more of the other ports of the network- 
ing device 200. In other embodiments, the networking device 
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200 implements various mixes of protocol layering and func- 
tionality, and can include functionality not conforming to 
standard communication models. 

In one exemplary implementation of the embodiment 
shown in FIG. 2, the network 201 comprises a local area 
network used in or with a satellite or other vehicle (such as an 
automobile, aircraft, and/or space craft), where the nodes 
203-1 . . . 203 -N are implemented as sensors, processing 
units, memory devices, etc. associated with the operation of 
such a vehicle. Other embodiments are implemented in other 
ways and for other applications. 

Device 200 includes circuit 204-1 and circuit 204-2. For 
each of port 205, device 200 includes a respective command 
module 224 (also referred to herein as “COM” or “command” 
hardware and individually labeled 224-1, . . . , 224-N) and a 
monitor module 226 (also referred to herein as “MON” or 
“monitor” hardware and individually labeled 226-1, . . . , 
226-N). Each command module 224 comprises a transceiver 
having a transmitter 210 and a receiver 212. Each monitor 
module 226 comprises two transceivers, each of which 
including a respective transmitter 210 and a receiver 212. 
Unlike the device 100 shown in FIG. 1, both command mod- 
ules 224 and monitor modules 226 are located in each of 
circuits 204-1 and 204-2. In particular, for each port 205, the 
respective command module 224 is located in one of the 
circuits 204-1 or 204-2 while the corresponding monitor 
module 226 is located in the other circuit 204-2 or 204-1. In 
other words, for each port 205 with a command module 224 
located in circuit 204-1, that command module 224 is moni- 
tored by a corresponding monitor module 226 in circuit 204- 
2. Similarly, for each port 205 with a command module 224 
located in circuit 204-2, that command module 224 is moni- 
tored by a corresponding monitor module 226 in circuit 204- 
1 . 

For each port 205, the transmitter 210 in that port’s com- 
mand module 224 is coupled to the transmit portion 214 of the 
communication channel 218 to which that port 205 is 
coupled. Similarly, for each port 205, the receiver 212 in that 
port’ s command module 224 is coupled to the receive portion 
216 of the communication channel 218 to which that port 205 
is coupled. Each port’ s monitor module 226 has two receivers 
212: one coupled to the transmitter 210 of the corresponding 
command module 224 and another coupled to the receive 
portion 216 of the communication channel 218 to which that 
port is coupled. 

Each communication channel is implemented using a suit- 
able communication medium or media (for example, metallic 
communication media such as twisted pair cables, coaxial 
cable and optical communication media such as fiber optic 
cables). In some embodiments, each communication channel 
also uses suitably partitioned (for example, via code, fre- 
quency, space, or time multiplexing) radio frequency (RF) or 
free- space optical paths. The transmit portion 214 of each 
communication channel 218 is the communication medium 
or media over which data is transmitted. Likewise, the receive 
portion 216 of each communication channel 218 is the com- 
munication medium or media from which data is received. In 
some embodiments, the transmit portion 214 and the receive 
portion 216 of the communication channel 218 is imple- 
mented using a separate communication medium or media. 
For example, in one embodiment where the communication 
channel 218 is implemented using a copper twisted-pair cable 
such as a category 5 (“CAT 5”) cable, one pair of copper wires 
included in the CAT5 cable is used for transmitting data and 
another pair of copper wires included in the CAT5 cable is 
used for receiving data. In such an embodiment, the former 
pair of copper wires comprises the transmit portion 214 of the 
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communication channel 218, and the latter pair of copper 
wires comprises the receive portion 216 of the communica- 
tion channel 218. In another embodiment where the commu- 
nication channel 218 is implemented using fiber optic cables, 
5 one optical cable is used to transmit data for that communi- 
cation channel 218 and another optical cable is used to receive 
data from that communication channel 218. In such an 
embodiment, the first optical fiber comprises the transmit 
portion 214 of the communication channel 218 and the sec- 
10 ond optical fiber comprises the receive portion 216 of the 
communication channel 218. In other embodiments, the 
transmit portion 214 and the receive portion 216 of the com- 
munication channel 218 are implemented using the same 
communication medium or media, for example, by using a 
15 multiplexing technique such as time division multiplexing, 
frequency division multiplexing, wave division multiplexing, 
or code division multiplexing. In some embodiments, each 
transmit portion and each receive portion of a communication 
channel uses multiple segments of media in parallel. 

20 In some embodiments, it is necessary for the circuit 204-1 
to communicate with circuit 204-2. For example, communi- 
cation between circuits 204-1 and 204-2 is necessary in some 
embodiments to transfer data received on port 205-1 by cir- 
cuit 204-1 to circuit 204-2 for transmitting on port 205-N. 
25 Exemplary situations in which communication between cir- 
cuits 204 is necessary or desired include embodiments com- 
prising three or more circuits 204 or embodiments wherein 
each transmitter can have only one receiver (such as for 
quantum cryptography ports 205 with the quantum devices 
30 embedded in each circuit 204). In such embodiments, an 
inter-chip link 234 is used to facilitate the necessary commu- 
nication between circuits 204-1 and 204-2. However, it is to 
be understood that inter-chip link 234 is not necessary in 
some embodiments since each of circuits 204-1 and 204-2 
35 receive a copy of all inputs on ports 205-1 . . . 205-N. In 
addition, in embodiments utilizing inter-chip link 234, a cir- 
cuit having a monitor module 226 is precluded from provid- 
ing information via the inter-chip link 234 which influences 
the output of the command module 224 being monitored. 
40 Similarly, a circuit having a command module 224 is pre- 
cluded from providing information via the inter-chip link 234 
which adversely impacts the ability of the corresponding 
monitor module 226 to monitor the command module 224. 

Although an embodiment is shown in FIG. 2 in which the 
45 COM/MON techniques described here are used to implement 
a networking device, it is to be understood that such tech- 
niques can be used in other embodiments. For example, the 
COM/MON techniques described here can be used to imple- 
ment terminal nodes. One example of such an embodiment is 
50 shown in FIG. 3. In the embodiment shown in FIG. 3, sub- 
stantially the same COM/MON functionality described 
above in connection with the embodiment shown in FIG. 2 is 
used to implement the COM/MON functionality of terminal 
node 300 (and the same reference numerals are used in FIG. 
55 3 as in FIG. 2 to refer to substantially similar functionality, the 
description of which is not repeated here for clarity). The 
terminal node 300 typically will have fewer ports 205. In the 
particular embodiment shown in FIG. 3, the terminal node 
300 comprises two ports 205, each of which is coupled to a 
60 different networking device 302 via a respective communi- 
cation channel 218. 

By placing both command modules 224 and monitor mod- 
ules 226 in both circuits 204-1 and 204-2, embodiments of the 
devices 200 and 300 reduce waste typically associated with 
65 conventional devices, such as device 100. In particular, the 
embodiments described in FIGS. 2 and 3 reduce the waste of 
a transceiver for each port. This reduction in waste enables 
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lower production costs and/or the ability to provide support coupled to receive data and instructions from, and to transmit 

for more ports than a conventional device of similar size. The data and instructions to, a data storage system, at least one 

additional ports are a product of being able to use the trans- input device, and at least one output device. Generally, a 

mitters and receivers wasted in conventional devices to pro- processor will receive instructions and data from a read-only 

vide additional ports. In addition, as shown in FIGS. 2 and 3, 5 memory and/or a random access memory. Storage devices 
circuits 204-2 and 204-1 share identical designs, in some suitable for tangibly embodying computer program instruc- 

embodiments, which further reduces development and/or tions and data include all forms of non-volatile memory, 

logistics costs. including by way of example semiconductor memory 

Furthermore, although embodiments are shown in FIGS. 2 devices, such as EPROM, EEPROM, and flash memory 

and 3 in which the COM/MON techniques described here are lo devices; magnetic disks such as internal hard disks and 
used in devices implemented with two circuits 204-1 and removable disks; magneto-optical disks; and DVD disks. Any 

204-2, it is to be understood that such techniques can be used of the foregoing may be supplemented by, or incorporated in, 

in other embodiments. For example, the COM/MON tech- specially-designed application- specific integrated circuits 

niques described here can be used to implement devices hav- (ASICs), field programmable gate arrays (FPGAs), or other 

ing more than two circuits 204-1 and 204-2. In the embodi- 15 circuitry. 

ment shown in FIG. 4, substantially the same COM/MON A number of embodiments of the invention defined by the 

functionality described above in connection with the embodi- following claims have been described. Nevertheless, it will be 

ment shown in FIG. 2 is used to implement the COM/MON understood that various modifications to the described 

functionality of device 400 (and the same reference numerals embodiments may be made without departing from the spirit 

are used in FIG. 4 as in FIG. 2 to refer to substantially similar 20 and scope of the claimed invention. Accordingly, other 
functionality, the description of which is not repeated here for embodiments are within the scope of the following claims, 

clarity). However, device 400 includes three COM/MON cir- What is claimed is: 

cuits 204-1, 204-2, and 204-3. 1. A communication device comprising: 

Circuits 204-1 . . . 204-3 monitor one another as described first and second circuits to implement a plurality of ports 

above. In particular, as shown in FIG. 4, command module 25 via which the communicative device is operable to com- 
224-1 in circuit 204-1 is monitored by monitor module 226-1 municate over a plurality of communication channels; 

in circuit 204-2. Similarly, command module 224-2 in circuit wherein, for each of the plurality of ports, the communi- 

204-2 is monitored by monitor module 226-2 in circuit 204-3 . cation device comprises: 

Finally, command module 224-3 in circuit 204-3 is monitored command hardware that includes a first transmitter to 

by monitor module 226-3 in circuit 204-1. In addition, circuit 30 transmit data over a respective one of the plurality of 

204-1 is coupled to circuit 204-3 via an inter-chip link 234 in channels and a first receiver to receive data from the 

order to facilitate communication between circuits 204-1 and respective one of the plurality of channels; and 

204-3 when necessary, e.g. to transfer data from port 205-2 to monitor hardware that includes a second receiver 

port 205-1, circuit 204-3 sends the data from port 205-2 to coupled to the first transmitter to receive data from the 

circuit 204-1 so that its command module 224-1 can transmit 35 first transmitter and a third receiver coupled to the 

the data to port 205-1. respective one of the plurality of channels, wherein 

Similarly, circuit 204-3 is coupled to circuit 204-2 and the third receiver receives data from the respective 

circuit 204-1 is coupled to circuit 204-2 via an inter-chip link one of the plurality of channels; 

234 to facilitate communication between circuits 204-1, wherein the first circuit comprises the command hardware 

204-2 and 204-3. Thus, by coupling chips 204-1 . . . 204-3 in 40 for a first subset of the plurality of ports; and 
a ring-like manner as shown in FIG. 4, circuits 204-1 ... 204-3 wherein the second circuit comprises the monitor hardware 

are able to monitor and communicate with one another. Nota- for the first subset of the plurality of ports and the com- 

bly, although only three circuits are shown in FIG. 4, it is to be mand hardware for a second subset of the plurality of 

understood that the pattern of coupling circuits 204-1 . . . ports. 

204-3 to monitor one another can be extended to include four 45 2. The communication device of claim 1, wherein the first 

or more circuits, each circuit monitoring another. In addition, circuit further comprises the monitor hardware for the second 

although the embodiment shown in FIG. 4 is described in subset of the plurality of ports. 

terms of a ring, it is to be understood that other embodiments 3. The communication device of claim 1, wherein the first 

may use other logical topologies in which each COM module circuit and the second circuit are substantially identical. 

224 still has an associated MON module 226 in an indepen- 50 4. The communication device of claim 1, further compris- 

dent circuit and each independent circuit contains a mix of ing an inter-chip link coupled to the first and second circuits 
COM and MON modules. and configured to enable communication between the first 

The methods and techniques described here may be imple- and second circuits, 

mented in digital electronic circuitry, or with a programmable 5 . The communication device of claim 1 , wherein the corn- 

processor (for example, a special-purpose processor or a gen- 55 munication device comprises a networking device, 
eral-purpose processor such as a computer) firmware, soft- 6. The communication device ofclaiml, wherein the corn- 

ware, or in combinations of them. Apparatus embodying munication device further comprises a third circuit, wherein 

these techniques may include appropriate input and output the third circuit comprises the command hardware for a third 

devices, a programmable processor, and a storage medium subset of the plurality of ports and the monitor hardware for 
tangibly embodying program instructions for execution by 60 the second subset of the plurality of ports, 
the programmable processor. A process embodying these 7. A system comprising: 

techniques may be performed by a programmable processor a plurality of nodes; and 

executing a program of instructions to perform desired func- at least one networking device communicatively coupling 

tions by operating on input data and generating appropriate the plurality of nodes to one another; 

output. The techniques may advantageously be implemented 65 wherein the networking device comprises a plurality of 
in one or more programs that are executable on a program- ports to communicatively couple the networking device 

mable system including at least one programmable processor to a plurality of communication channels, wherein each 
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of the plurality of nodes is communicatively coupled to 
the networking device via at least one of the plurality of 
ports; 

wherein the networking device further comprises first and 
second circuits; ^ 

wherein, for each of the plurality of ports, the networking 
device comprises: 

command hardware that includes a first transmitter to 
transmit data over a respective one of the plurality of 
channels and a first receiver to receive data from the 
respective one of the plurality of channels; 
monitor hardware that includes a second receiver 
coupled to the first transmitter to receive data from the 
first transmitter and a third receiver coupled to the 15 
respective one of the plurality of channels, wherein 
the third receiver receives data from the respective 
one of the plurality of channels; 

wherein the first circuit comprises the command hardware 
for a first subset of the plurality of ports; and 20 

wherein the second circuit comprises the monitor hardware 
for the first subset of the plurality of ports and the com- 
mand hardware for a second subset of the plurality of 
ports. 

25 

8 . The system of claim 7, wherein the first circuit further 
comprises the monitor hardware for the second subset of the 
plurality of ports. 

9. The system of claim 7, wherein the first circuit and the 

second circuit are substantially identical. 3 q 

10. The system of claim 7, further comprising an inter-chip 
link coupled to the first and second circuits and configured to 
enable communication between the first and second circuits. 

11. The system of claim 7, wherein the networking device 
further comprises a third circuit wherein the third circuit 
comprises the command hardware for a third subset of the 
plurality of ports and the monitor hardware for the second 
subset of the plurality of ports. 

12. The system of claim 7, wherein the networking device 
is communicatively coupled to the plurality of nodes via one 
of twisted pair cable, coaxial cable, and fiber optic cable. 
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13. A communication device comprising: 

a plurality of ports to communicatively couple the commu- 
nication device to a plurality of communication chan- 
nels; 

a plurality of command modules to transmit and receive 
data on the plurality of communication channels, 
wherein each of the plurality of command modules com- 
prises a first transmitter to transmit data over a respective 
one of the plurality of channels and a first receiver to 
receive data from the respective one of the plurality of 
channels; 

a plurality of monitor modules to monitor data transmitted 
by the command modules and data received on the plu- 
rality of communication channels, wherein each of the 
plurality of monitor modules comprises a second 
receiver coupled to the first transmitter to receive data 
from the first transmitter and a third receiver coupled to 
the respective one of the plurality of channels, wherein 
the third receiver receives data from the respective one of 
the plurality of channels; 

wherein there is at least one command module and at least 
one monitor module for each of the plurality of ports; 
and 

wherein at least one of the command modules for a first 
subset of the plurality of ports are implemented in the 
same circuit as at least one of the monitor modules for a 
second subset of the plurality of ports. 

14. The communication device of claim 13, wherein each 
monitor module is implemented in a different circuit than the 
command module being monitored by that monitor module. 

15. The communication device of claim 13, wherein the 
plurality of command modules and the plurality of monitor 
modules are implemented in two or more circuits, wherein 
each of the two or more circuits comprises at least one of the 
plurality of command modules and at least one of the plurality 
of monitor modules. 

16. The communication device of claim 15, wherein two or 
more circuits are substantially identical circuits. 

17. The communication device of claim 15, further com- 
prising an inter-chip link coupled to the two or more circuits 
and configured to enable communication between the two or 
more circuits. 





